Stereo-Vision Based 3D Modeling for Unmanned Ground Vehicles

Instant Scene Modeler (iSM) is a vision system for generating calibrated photo-realistic 3D models of unknown environments quickly using stereo image sequences. Equipped with iSM, Unmanned Ground Vehicles (UGVs) can capture stereo images and create 3D models to be sent back to the base station, while they explore unknown environments. Rapid access to 3D models will increase the operator situational awareness and allow better mission planning and execution, as the models can be visualized from different views and used for relative measurements. Current military operations of UGVs in urban warfare threats involve the operator hand-sketching the environment from live video feed. iSM eliminates the need for an additional operator as the 3D model is generated automatically. The photo-realism of the models enhances the situational awareness of the mission and the models can also be used for change detection. iSM has been tested on our autonomous vehicle to create photo-realistic 3D models while the rover traverses in unknown environments. Moreover, a proof-of-concept iSM payload has been mounted on an iRobot PackBot with Wayfarer technology, which is equipped with autonomous urban reconnaissance capabilities. The Wayfarer PackBot UGV uses wheel odometry for localization and builds 2D occupancy grid maps from a laser sensor. While the UGV is following walls and avoiding obstacles, iSM captures and processes images to create photo-realistic 3D models. Experimental results show that iSM can complement Wayfarer PackBot’s autonomous navigation in two ways. The photo-realistic 3D models provide better situational awareness than 2D grid maps. Moreover, iSM also recovers the camera motion, also known as the visual odometry. As wheel odometry error grows over time, this can help improve the wheel odometry for better localization.